Papers by Luis Fernando D’Haro
DynaEval: Unifying Turn and Dialogue Level Evaluation (2021.acl-long)
Copied to clipboard
Chen Zhang, Yiming Chen, Luis Fernando D’Haro, Yan Zhang, Thomas Friedrichs, Grandee Lee, Haizhou Li
| Challenge: | Existing evaluation metrics focus on the turn-level quality of a dialogue . a unified framework that holistically considers the quality of the entire dialogue is needed . |
| Approach: | They propose a unified automatic evaluation framework which holistically considers the quality of the entire dialogue. |
| Outcome: | The proposed framework outperforms the state-of-the-art dialogue coherence model and correlates strongly with human judgements across multiple evaluation aspects at both turn and dialogue level. |
Unveiling the Achilles’ Heel of NLG Evaluators: A Unified Adversarial Framework Driven by Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have highlighted various neural metrics that align well with human evaluations. |
| Approach: | They propose a black-box adversarial framework that generates strong disagreements between human and victim evaluators. |
| Outcome: | The proposed framework can significantly improve the performance of human and victim evaluators. |
Beyond Single-Audio: Advancing Multi-Audio Processing in Audio Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluations of audio large language models focus on single audio inputs, but real-world applications often require processing multiple audio streams simultaneously. |
| Approach: | They propose a multi-audio evaluation benchmark that combines 20 audio inputs from 11 audio tasks to capture audio context. |
| Outcome: | The proposed model outperforms baseline models and achieves high data efficiency without human annotations. |
FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent model-based reference-free metrics for open-domain dialogue evaluation lack correlations with human judgment and poor interpretability. |
| Approach: | They propose a multi-dimensional dialogue-level metric with three sub-metrics targeting a specific dimension. |
| Outcome: | The proposed metric outperforms existing models and sub-metrics in three high-quality dialogue evaluation benchmarks. |